Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

InterceptMessage cleanup #1339

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

InterceptMessage cleanup #1339

wants to merge 3 commits into from

Conversation

plebhash
Copy link
Collaborator

@plebhash plebhash commented Jan 9, 2025

close #1338

@plebhash plebhash changed the title Intercept message cleanup InterceptMessage cleanup Jan 9, 2025
@plebhash plebhash force-pushed the intercept-message-cleanup branch from b5e6e91 to 85433a8 Compare January 9, 2025 14:08
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintercept-message-cleanup
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,524.90
(-0.47%)		6,932.74
(94.12%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,368.80
(-0.53%)		7,842.92
(93.95%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		7,985.70
(-1.33%)		9,383.76
(85.10%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		850.33
(-1.93%)		942.13
(90.26%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		668.61
(-1.07%)		723.45
(92.42%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		246.79
(-1.25%)		273.62
(90.19%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		157.71
(+0.14%)		165.16
(95.49%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,249.70
(-1.47%)		6,776.43
(92.23%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		273.96
(-3.41%)		336.73
(81.36%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		719.38
(-1.05%)		778.55
(92.40%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		577.59
(-2.13%)		628.84
(91.85%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		203.71
(-1.27%)		226.11
(90.09%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintercept-message-cleanup
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
1e3 x instructions
(Result Δ%)		Upper Boundary
1e3 x instructions
(Limit %)		L1 AccessesBenchmark Result
1e3 x accesses
(Result Δ%)		Upper Boundary
1e3 x accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8.39
(-0.79%)		8.67
(96.71%)		📈 view plot
🚷 view threshold		3.69
(-0.93%)		3.86
(95.53%)		📈 view plot
🚷 view threshold		5.16
(-1.06%)		5.45
(94.68%)		📈 view plot
🚷 view threshold		9.00
(+5.59%)		16.42
(54.80%)		📈 view plot
🚷 view threshold		91.00
(-0.45%)		96.82
(93.99%)
get_submit📈 view plot
🚷 view threshold		95.18
(-0.18%)		95.61
(99.55%)		📈 view plot
🚷 view threshold		59.35
(-0.09%)		59.71
(99.39%)		📈 view plot
🚷 view threshold		85.23
(-0.10%)		85.82
(99.31%)		📈 view plot
🚷 view threshold		45.00
(+0.40%)		60.35
(74.56%)		📈 view plot
🚷 view threshold		278.00
(-0.98%)		291.88
(95.24%)
get_subscribe📈 view plot
🚷 view threshold		7.86
(-1.61%)		8.23
(95.55%)		📈 view plot
🚷 view threshold		2.77
(-1.65%)		2.94
(93.97%)		📈 view plot
🚷 view threshold		3.85
(-1.85%)		4.14
(92.92%)		📈 view plot
🚷 view threshold		12.00
(-4.36%)		20.73
(57.87%)		📈 view plot
🚷 view threshold		113.00
(-1.33%)		117.94
(95.81%)
serialize_authorize📈 view plot
🚷 view threshold		12.10
(-1.25%)		12.51
(96.79%)		📈 view plot
🚷 view threshold		5.27
(-0.59%)		5.43
(97.05%)		📈 view plot
🚷 view threshold		7.33
(-0.65%)		7.60
(96.41%)		📈 view plot
🚷 view threshold		10.00
(-2.66%)		18.89
(52.92%)		📈 view plot
🚷 view threshold		135.00
(-2.14%)		143.35
(94.18%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24.57
(-0.59%)		25.19
(97.54%)		📈 view plot
🚷 view threshold		9.84
(-0.19%)		10.01
(98.28%)		📈 view plot
🚷 view threshold		13.88
(-0.21%)		14.18
(97.90%)		📈 view plot
🚷 view threshold		38.00
(+5.65%)		45.95
(82.70%)		📈 view plot
🚷 view threshold		300.00
(-1.19%)		313.69
(95.64%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30.07
(-0.82%)		30.73
(97.87%)		📈 view plot
🚷 view threshold		12.02
(-0.28%)		12.19
(98.60%)		📈 view plot
🚷 view threshold		17.00
(-0.32%)		17.30
(98.30%)		📈 view plot
🚷 view threshold		59.00
(+5.59%)		67.50
(87.41%)		📈 view plot
🚷 view threshold		365.00
(-1.62%)		379.52
(96.17%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126.19
(-0.21%)		126.79
(99.53%)		📈 view plot
🚷 view threshold		73.20
(-0.06%)		73.53
(99.55%)		📈 view plot
🚷 view threshold		104.92
(-0.08%)		105.51
(99.44%)		📈 view plot
🚷 view threshold		110.00
(+3.02%)		125.90
(87.37%)		📈 view plot
🚷 view threshold		592.00
(-0.97%)		610.67
(96.94%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27.67
(-0.85%)		28.39
(97.49%)		📈 view plot
🚷 view threshold		9.58
(-0.47%)		9.76
(98.20%)		📈 view plot
🚷 view threshold		13.54
(-0.55%)		13.84
(97.83%)		📈 view plot
🚷 view threshold		69.00
(+6.80%)		78.07
(88.38%)		📈 view plot
🚷 view threshold		394.00
(-1.31%)		409.82
(96.14%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115.07
(-0.15%)		115.72
(99.44%)		📈 view plot
🚷 view threshold		68.06
(+0.01%)		68.42
(99.47%)		📈 view plot
🚷 view threshold		97.65
(+0.01%)		98.29
(99.35%)		📈 view plot
🚷 view threshold		67.00
(+2.07%)		87.76
(76.35%)		📈 view plot
🚷 view threshold		488.00
(-1.10%)		505.39
(96.56%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23.10
(-0.94%)		23.82
(97.00%)		📈 view plot
🚷 view threshold		8.14
(-0.51%)		8.32
(97.92%)		📈 view plot
🚷 view threshold		11.45
(-0.57%)		11.75
(97.51%)		📈 view plot
🚷 view threshold		41.00
(+4.98%)		51.15
(80.16%)		📈 view plot
🚷 view threshold		327.00
(-1.40%)		342.50
(95.48%)
serialize_submit📈 view plot
🚷 view threshold		99.57
(-0.22%)		100.09
(99.48%)		📈 view plot
🚷 view threshold		61.41
(-0.07%)		61.73
(99.47%)		📈 view plot
🚷 view threshold		88.08
(-0.08%)		88.65
(99.36%)		📈 view plot
🚷 view threshold		50.00
(+3.66%)		67.07
(74.55%)		📈 view plot
🚷 view threshold		321.00
(-1.36%)		337.65
(95.07%)
serialize_subscribe📈 view plot
🚷 view threshold		11.31
(-0.78%)		11.60
(97.51%)		📈 view plot
🚷 view threshold		4.12
(-1.03%)		4.28
(96.15%)		📈 view plot
🚷 view threshold		5.71
(-1.22%)		6.00
(95.20%)		📈 view plot
🚷 view threshold		14.00
(+0.11%)		24.30
(57.61%)		📈 view plot
🚷 view threshold		158.00
(-0.34%)		164.63
(95.97%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintercept-message-cleanup
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
1e3 x estimated cycles
(Result Δ%)		Upper Boundary
1e3 x estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2.14
(+1.04%)		2.24
(95.50%)		📈 view plot
🚷 view threshold		473.00
(-0.13%)		490.59
(96.41%)		📈 view plot
🚷 view threshold		732.00
(-0.54%)		758.98
(96.45%)		📈 view plot
🚷 view threshold		8.00
(+56.81%)		11.66
(68.58%)		📈 view plot
🚷 view threshold		39.00
(+0.85%)		41.73
(93.46%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8.36
(+1.56%)		8.40
(99.43%)		📈 view plot
🚷 view threshold		2,137.00📈 view plot
🚷 view threshold		3,151.00
(-0.24%)		3,167.58
(99.48%)		📈 view plot
🚷 view threshold		40.00
(+12.33%)		41.66
(96.02%)		📈 view plot
🚷 view threshold		143.00
(+2.34%)		144.65
(98.86%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6.39
(+1.46%)		6.45
(99.12%)		📈 view plot
🚷 view threshold		1,750.00
(-0.03%)		1,767.59
(99.00%)		📈 view plot
🚷 view threshold		2,545.00
(-0.27%)		2,575.49
(98.82%)		📈 view plot
🚷 view threshold		21.00
(+22.29%)		24.13
(87.03%)		📈 view plot
🚷 view threshold		107.00
(+2.17%)		108.84
(98.31%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14.86
(+0.92%)		14.94
(99.48%)		📈 view plot
🚷 view threshold		4,694.00
(-0.01%)		4,711.59
(99.63%)		📈 view plot
🚷 view threshold		6,742.00
(-0.20%)		6,786.80
(99.34%)		📈 view plot
🚷 view threshold		56.00
(+22.17%)		63.44
(88.28%)		📈 view plot
🚷 view threshold		224.00
(+1.26%)		226.21
(99.02%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27.86
(+0.92%)		28.07
(99.26%)		📈 view plot
🚷 view threshold		10,645.00
(+0.35%)		10,699.43
(99.49%)		📈 view plot
🚷 view threshold		15,500.00
(+0.38%)		15,595.29
(99.39%)		📈 view plot
🚷 view threshold		92.00
(+9.74%)		99.30
(92.65%)		📈 view plot
🚷 view threshold		340.00
(+1.32%)		343.30
(99.04%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4.46
(+1.16%)		4.58
(97.19%)		📈 view plot
🚷 view threshold		1,461.00
(-0.04%)		1,478.59
(98.81%)		📈 view plot
🚷 view threshold		2,155.00
(-0.26%)		2,184.76
(98.64%)		📈 view plot
🚷 view threshold		12.00
(+46.01%)		14.07
(85.30%)		📈 view plot
🚷 view threshold		64.00
(+1.71%)		67.79
(94.40%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14.05
(+0.23%)		14.20
(98.94%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,081.59
(99.65%)		📈 view plot
🚷 view threshold		7,317.00
(-0.12%)		7,352.80
(99.51%)		📈 view plot
🚷 view threshold		45.00
(+22.53%)		48.64
(92.52%)		📈 view plot
🚷 view threshold		186.00
(-0.01%)		190.89
(97.44%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22.93
(+0.98%)		23.08
(99.37%)		📈 view plot
🚷 view threshold		8,040.00
(+0.09%)		8,057.64
(99.78%)		📈 view plot
🚷 view threshold		11,684.00
(-0.01%)		11,713.47
(99.75%)		📈 view plot
🚷 view threshold		87.00
(+14.64%)		90.20
(96.45%)		📈 view plot
🚷 view threshold		309.00
(+1.58%)		312.95
(98.74%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4.71
(+0.49%)		4.79
(98.39%)		📈 view plot
🚷 view threshold		1,502.00
(-0.04%)		1,519.59
(98.84%)		📈 view plot
🚷 view threshold		2,275.00
(-0.16%)		2,300.98
(98.87%)		📈 view plot
🚷 view threshold		12.00
(+27.15%)		15.31
(78.37%)		📈 view plot
🚷 view threshold		68.00
(+0.58%)		70.14
(96.95%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16.23
(+0.45%)		16.33
(99.39%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,980.59
(99.71%)		📈 view plot
🚷 view threshold		8,650.00
(-0.16%)		8,692.74
(99.51%)		📈 view plot
🚷 view threshold		53.00
(+29.99%)		56.65
(93.56%)		📈 view plot
🚷 view threshold		209.00
(+0.35%)		212.17
(98.50%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35.76
(+0.42%)		35.96
(99.44%)		📈 view plot
🚷 view threshold		14,888.00
(+0.14%)		14,919.71
(99.79%)		📈 view plot
🚷 view threshold		21,866.00
(+0.10%)		21,919.07
(99.76%)		📈 view plot
🚷 view threshold		111.00
(+17.26%)		120.22
(92.33%)		📈 view plot
🚷 view threshold		381.00
(+0.36%)		385.75
(98.77%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025 •
edited
Loading

🐰 Bencher Report

Branchintercept-message-cleanup
Testbedsv2

🚨 1 Alert

BenchmarkMeasure
Units		ViewBenchmark Result
(Result Δ%)		Upper Boundary
(Limit %)
client_sv2_mining_message_submit_standardLatency
nanoseconds (ns)		📈 plot
🚨 alert (🔔)
🚷 threshold		15.07
(+2.62%)		15.03
(100.24%)
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		45.24
(-0.02%)		59.60
(75.91%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		75.86
(-1.88%)		106.70
(71.10%)
client_sv2_mining_message_submit_standard📈 view plot
🚨 view alert (🔔)
🚷 view threshold		15.07
(+2.62%)		15.03
(100.24%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		259.70
(-1.58%)		288.17
(90.12%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		640.63
(+3.56%)		683.64
(93.71%)
client_sv2_open_channel📈 view plot
🚷 view threshold		171.11
(+3.00%)		180.00
(95.06%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		294.09
(+2.67%)		315.43
(93.23%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		395.36
(+3.14%)		405.47
(97.51%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		170.61
(+6.34%)		174.79
(97.61%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		497.99
(+5.80%)		558.36
(89.19%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		1,115.60
(+10.07%)		1,241.52
(89.86%)
🐰 View full continuous benchmarking report in Bencher

@codecov Codecov
Copy link

codecov bot commented Jan 9, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.14%. Comparing base (a053dc7) to head (f7f58fd).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1339   +/-   ##
=======================================
  Coverage   19.14%   19.14%           
=======================================
  Files         166      166           
  Lines       10987    10987           
=======================================
  Hits         2104     2104           
  Misses       8883     8883
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.48% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (ø)
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.49% <ø> (ø)
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 2.05% <ø> (ø)
protocols 24.74% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.11% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils ?
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@plebhash plebhash mentioned this pull request Jan 9, 2025
Open
jbesraa
jbesraa approved these changes Jan 9, 2025
View reviewed changes
Copy link
Contributor

@jbesraa jbesraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just added a nit

roles/tests-integration/lib/sniffer.rs Outdated
@@ -47,10 +47,10 @@ enum SnifferError {
/// In order to alter the messages sent between the roles, the [`Sniffer::intercept_messages`]
/// field can be used. It will look for the [`InterceptMessage::expected_message_type`] in the
/// specified [`InterceptMessage::direction`] and replace it with
/// [`InterceptMessage::response_message`].
/// [`InterceptMessage::replace_message`].
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea in practice response_message can also be request_message, I agree this is confusing.

Maybe we should use replacement_message? replace_message feels more like a function name.

Copy link
Collaborator Author

@plebhash plebhash Jan 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rebased

this confusion is also clarified in docs via #1340 (more specifically b193189), as the description of Sniffer was also based on a similar assumption

roles/tests-integration/lib/sniffer.rs
@@ -80,14 +75,12 @@ impl InterceptMessage {
expected_message_type: MsgType,
replace_message: PoolMessages<'static>,
replace_message_type: MsgType,
break_on: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont mind removing this tbh. It was added because it can be useful to decide to stop the connection between two roles when a specific message is exchanged. Some of those cases are now covered by the check_on_drop flag thus I think it is cool to remove it.

@plebhash
rename InterceptMessage.response_message to replacement_message..
1a188b6 
this is a misleading name, as this is not really a response, but rather the new message to replace the intercepted one
@plebhash
remove interrupt functionality from Sniffer..
c7eeeaa 
it's not fully designed and motivation is not clear, we can re-design if needed
@plebhash
expand test_sniffer_intercept to make test comprehensive..
f7f58fd 
we need 2 sniffers to assert the message was correctly replaced and fully delivered over the TCP connection
@plebhash plebhash force-pushed the intercept-message-cleanup branch from 85433a8 to f7f58fd Compare January 9, 2025 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbesraa jbesraa jbesraa approved these changes

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

need Integration Tests InterceptMessage cleanup
2 participants
@plebhash @jbesraa